Skip to content

feat(dns): NSX DNS record service and zone validation#1423

Open
wenyingd wants to merge 1 commit into
vmware-tanzu:mainfrom
wenyingd:dns-only
Open

feat(dns): NSX DNS record service and zone validation#1423
wenyingd wants to merge 1 commit into
vmware-tanzu:mainfrom
wenyingd:dns-only

Conversation

@wenyingd
Copy link
Copy Markdown
Contributor

@wenyingd wenyingd commented May 6, 2026

  1. Implement DNSRecordService for NSX ProjectDnsRecord CRUD operations
  2. Validate hostnames against VPCNetworkConfiguration allowed DNS zones
  3. Wrap hostname-mismatch error as DNSZoneValidationError for accurate
    DNSRecordReady condition reporting
  4. Remove the realization state check before NSX API is ready.

Testing Done:

  1. Prepare a DNS zone in NSX as below,
{
  "dns_domain_name": "example.com",
  "ttl": 300,
  "resource_type": "ProjectDnsZone",
  "id": "project-zone",
  "display_name": "project-zone",
  "path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
  "relative_path": "project-zone",
  "parent_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service",
  "remote_path": "",
  "unique_id": "1e159072-29c4-469d-adee-7b460dd4c640",
  "realization_id": "1e159072-29c4-469d-adee-7b460dd4c640",
  "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
  "marked_for_delete": false,
  "overridden": false,
  "_system_owned": false,
  "_protection": "NOT_PROTECTED",
  "_create_time": 1779435122113,
  "_create_user": "admin",
  "_last_modified_time": 1779435122113,
  "_last_modified_user": "admin",
  "_revision": 0
}
  1. Create a LB Service , and annotate it with annotation
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE
gateway-istio   LoadBalancer   172.24.184.123   192.168.0.8   15021:31957/TCP,80:32095/TCP   21m
httpbin         ClusterIP      172.24.60.20     <none>        8000/TCP                       21m
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.example.com"
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.example.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1783567"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:00:26Z"
    message: ""
    reason: DNSRecordConfigured
    status: "True"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check DNS record in NSX policy API, the following records are returned,
{
  "results": [
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "my-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "my-svc_project-zone_a",
      "display_name": "my-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/my-svc_project-zone_a",
      "relative_path": "my-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "realization_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779447625667,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779447625667,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    }
  ],
  "result_count": 1,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Add multiple hostnames in the service with annotation
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.example.com,istio-gw-svc.example.com" --overwrite
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.example.com,istio-gw-svc.example.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1792158"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:00:26Z"
    message: ""
    reason: DNSRecordConfigured
    status: "True"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, two records are returned,
{
  "results": [
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "istio-gw-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "istio-gw-svc_project-zone_a",
      "display_name": "istio-gw-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/istio-gw-svc_project-zone_a",
      "relative_path": "istio-gw-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "5ab46632-d8f9-45a8-a036-2d7f6aa378a1",
      "realization_id": "5ab46632-d8f9-45a8-a036-2d7f6aa378a1",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779448227678,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779448227678,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    },
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "my-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "my-svc_project-zone_a",
      "display_name": "my-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/my-svc_project-zone_a",
      "relative_path": "my-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "realization_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779447625667,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779447625667,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    }
  ],
  "result_count": 2,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Remove the annotation from the LB Service,
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname-
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# 
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# 
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1794628"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, all records are removed.
{
  "results": [],
  "result_count": 0,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Annotate the Service with a hostname not matching any permitted DNS domains,
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.invalid-zone.com"
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.invalid-zone.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1795871"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:14:50Z"
    message: hostname "my-svc.invalid-zone.com" does not match any allowed DNS domain
      in the namespace
    reason: DNSRecordFailed
    status: "False"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, no records are created
{
  "results": [],
  "result_count": 0,
  "sort_by": "display_name",
  "sort_ascending": true
}

@wenyingd wenyingd requested a review from TaoZou1 May 6, 2026 04:59
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 6, 2026

Codecov Report

❌ Patch coverage is 89.85782% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.60%. Comparing base (934da6b) to head (c5a41f0).

Files with missing lines Patch % Lines
pkg/nsx/services/dns/recordservice.go 82.28% 26 Missing and 19 partials ⚠️
pkg/nsx/services/dns/store.go 94.19% 7 Missing and 6 partials ⚠️
pkg/nsx/services/dns/compare.go 84.00% 9 Missing and 3 partials ⚠️
pkg/nsx/services/dns/zones.go 88.77% 6 Missing and 5 partials ⚠️
pkg/nsx/services/common/wrap.go 73.33% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/builder.go 94.73% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/cleanup.go 73.33% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/contributing.go 94.59% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/initialize.go 88.57% 2 Missing and 2 partials ⚠️
pkg/clean/clean.go 71.42% 1 Missing and 1 partial ⚠️
... and 2 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1423      +/-   ##
==========================================
+ Coverage   77.00%   77.60%   +0.59%     
==========================================
  Files         156      169      +13     
  Lines       22205    23254    +1049     
==========================================
+ Hits        17100    18047     +947     
- Misses       3892     3950      +58     
- Partials     1213     1257      +44     
Flag Coverage Δ
unit-tests 77.60% <89.85%> (+0.59%) ⬆️
Files with missing lines Coverage Δ
pkg/nsx/client.go 93.42% <100.00%> (+0.11%) ⬆️
pkg/nsx/services/common/policy_tree.go 86.48% <100.00%> (+0.24%) ⬆️
pkg/nsx/services/common/types.go 100.00% <ø> (ø)
pkg/nsx/services/dns/errors.go 100.00% <100.00%> (ø)
pkg/nsx/services/dns/types.go 100.00% <100.00%> (ø)
pkg/third_party/externaldns/endpoint/endpoint.go 100.00% <100.00%> (ø)
pkg/util/utils.go 87.12% <100.00%> (+0.12%) ⬆️
pkg/clean/clean.go 86.88% <71.42%> (-0.95%) ⬇️
pkg/third_party/externaldns/endpoint/utils.go 90.47% <90.47%> (ø)
pkg/third_party/externaldns/provider/zonefinder.go 90.90% <90.90%> (ø)
... and 9 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wenyingd wenyingd force-pushed the dns-only branch 8 times, most recently from 0825801 to 7f460a0 Compare May 12, 2026 02:16
@wenyingd wenyingd force-pushed the dns-only branch 4 times, most recently from 944694e to d5459d9 Compare May 14, 2026 11:40
TaoZou1
TaoZou1 previously approved these changes May 15, 2026
Comment thread pkg/nsx/services/dns/recordservice.go
@TaoZou1
Copy link
Copy Markdown
Contributor

TaoZou1 commented May 15, 2026

Please add enough UT later.

@wenyingd wenyingd force-pushed the dns-only branch 2 times, most recently from ebdc0d8 to 236e7bf Compare May 15, 2026 06:01
@wenyingd wenyingd requested a review from TaoZou1 May 15, 2026 06:44
@wenyingd wenyingd force-pushed the dns-only branch 2 times, most recently from 2593094 to c882eb5 Compare May 15, 2026 07:55
TaoZou1
TaoZou1 previously approved these changes May 15, 2026
@wenyingd
Copy link
Copy Markdown
Contributor Author

/e2e

@dantingl dantingl requested a review from heypnus May 18, 2026 02:20
@heypnus
Copy link
Copy Markdown
Contributor

heypnus commented May 21, 2026

/e2e

- Implement DNSRecordService for NSX ProjectDnsRecord CRUD operations
- Validate hostnames against VPCNetworkConfiguration allowed DNS zones
- Wrap hostname-mismatch error as DNSZoneValidationError for accurate
  DNSRecordReady condition reporting
- Remove the realization state check before NSX API is ready
@wenyingd
Copy link
Copy Markdown
Contributor Author

@TaoZou1 @heypnus I removed the logic with realization state check for the NSX API is not ready yet.

Testing Done section is added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants